Study on method of organ section retention and tracking through deep learning in automated diagnostic and therapeutic robotics.

PURPOSE: In high-intensity focused ultrasound (HIFU) treatment of the kidney and liver, tracking the organs is essential because respiratory motions make continuous cauterization of the affected area difficult and may cause damage to other parts of the body. In this study, we propose a tracking system for rotational scanning, and propose and evaluate a method for estimating the angles of organs in ultrasound images. METHODS: We proposed AEMA, AEMAD, and AEMAD++ as methods for estimating the angles of organs in ultrasound images, using RUDS and a phantom to acquire 90-degree images of a kidney from the long-axis image to the short-axis image as a data set. Six datasets were used, with five for preliminary preparation and one for testing, while the initial position was shifted by 2 mm in the contralateral axis direction. The test data set was evaluated by estimating the angle using each method. RESULTS: The accuracy and processing speed of angle estimation for AEMA, AEMAD, and AEMAD++ were 23.8% and 0.33 FPS for AEMAD, 32.0% and 0.56 FPS for AEMAD, and 29.5% and 3.20 FPS for AEMAD++, with tolerance of ± 2.5 degrees. AEMAD++ offered the best speed and accuracy. CONCLUSION: In the phantom experiment, AEMAD++ showed the effectiveness of tracking the long-axis image of the kidney in rotational scanning. In the future, we will add either the area of surrounding organs or the internal structure of the kidney as a new feature to validate the results.

A VS ultrasound diagnostic system with kidney image evaluation functions.

PURPOSE: An inevitable feature of ultrasound-based diagnoses is that the quality of the ultrasound images produced depends directly on the skill of the physician operating the probe. This is because physicians have to constantly adjust the probe position to obtain a cross section of the target organ, which is constantly shifting due to patient respiratory motions. Therefore, we developed an ultrasound diagnostic robot that works in cooperation with a visual servo system based on deep learning that will help alleviate the burdens imposed on physicians. METHODS: Our newly developed robotic ultrasound diagnostic system consists of three robots: an organ tracking robot (OTR), a robotic bed, and a robotic supporting arm. Additionally, we used different image processing methods (YOLOv5s and BiSeNet V2) to detect the target kidney location, as well as to evaluate the appropriateness of the obtained ultrasound images (ResNet 50). Ultimately, the image processing results are transmitted to the OTR for use as motion commands. RESULTS: In our experiments, the highest effective tracking rate (0.749) was obtained by YOLOv5s with Kalman filtering, while the effective tracking rate was improved by about 37% in comparison with cases without such filtering. Additionally, the appropriateness probability of the ultrasound images obtained during the tracking process was also the highest and most stable. The second highest tracking efficiency value (0.694) was obtained by BiSeNet V2 with Kalman filtering and was a 75% improvement over the case without such filtering. CONCLUSION: While the most efficient tracking achieved is based on the combination of YOLOv5s and Kalman filtering, the combination of BiSeNet V2 and Kalman filtering was capable of detecting the kidney center of gravity closer to the kidney's actual motion state. Furthermore, this model could also measure the cross-sectional area, maximum diameter, and other detailed information of the target kidney, which meant it is more practical for use in actual diagnoses.

Robust and fast laparoscopic vision-based ultrasound probe tracking using a binary dot array marker.

Laparoscopic vision-based ultrasound probe tracking systems have gained considerable attention in ultrasound-guided laparoscopic surgeries as replacements for external tracking systems (e.g. optical tracking and electromagnetic tracking systems), which increase cost and setting time, require additional operation space, and introduce new limitations. Most existing laparoscopic ultrasound (LUS) probe tracking systems rely on fiducial markers, which cannot easily realise fast and robust vision-based tracking in laparoscopic surgery owing to their design limitations. Therefore, we propose a novel binary dot array marker to realise a robust and fast LUS probe tracking system. The binary dot array marker comprises two dots (green and blue), which form multiple unique identification dot subarrays in the binary dot array. The binary dot array marker can be tracked when one of the identification dot subarrays is detected and identified; this novel design makes the binary dot array marker-based probe tracking system robust against occlusions during surgery. The evaluation results indicate that the proposed binary dot marker performs better in terms of robustness, computational efficiency, and tracking accuracy compared to the state-of-the-art fiducial markers used for vision-based probe tracking.

Automatic Aortic Valve Cusps Segmentation from CT Images Based on the Cascading Multiple Deep Neural Networks.

Accurate morphological information on aortic valve cusps is critical in treatment planning. Image segmentation is necessary to acquire this information, but manual segmentation is tedious and time consuming. In this paper, we propose a fully automatic aortic valve cusps segmentation method from CT images by combining two deep neural networks, spatial configuration-Net for detecting anatomical landmarks and U-Net for segmentation of aortic valve components. A total of 258 CT volumes of end systolic and end diastolic phases, which include cases with and without severe calcifications, were collected and manually annotated for each aortic valve component. The collected CT volumes were split 6:2:2 for the training, validation and test steps, and our method was evaluated by five-fold cross validation. The segmentation was successful for all CT volumes with 69.26 s as mean processing time. For the segmentation results of the aortic root, the right-coronary cusp, the left-coronary cusp and the non-coronary cusp, mean Dice Coefficient were 0.95, 0.70, 0.69, and 0.67, respectively. There were strong correlations between measurement values automatically calculated based on the annotations and those based on the segmentation results. The results suggest that our method can be used to automatically obtain measurement values for aortic valve morphology.

A novel complementation method of an acoustic shadow region utilizing a convolutional neural network for ultrasound-guided therapy.

PURPOSE: Noise-free ultrasound images are essential for organ monitoring during regional ultrasound-guided therapy. When the affected area is located under the ribs, however, acoustic shadow is caused by the reflection of sound from hard tissues such as bone, and the image is output with missing information in this region. Therefore, in the present study, we attempt to complement the image in the missing area. METHODS: The overall flow of the complementation method to generate a shadow-free composite image is as follows. First, we constructed a binary classification method for the presence or absence of acoustic shadow on a phantom kidney based on a convolutional neural network. Second, we created a composite shadow-free image by searching for a suitable image from a time-series database and superimposing the corresponding area without shadow onto the missing area of the target image. In addition, we constructed and verified an automatic kidney mask generation method utilizing U-Net. RESULTS: The complementation accuracy for kidney tracking could be enhanced by template matching. Zero-mean normalized cross-correlation (ZNCC) values after complementation were higher than that of before complementation under four different data generation conditions: (i) changing the position of the bed of the robotic ultrasound diagnostic system in the translational direction, (ii) changing the probe angle in the translational direction, (iii) with the addition of rotational motion of the probe to condition (ii). Although there was large variation in the shape of the kidney contour in condition (iii), the proposed method improved the ZNCC value from 0.5437 to 0.5807. CONCLUSIONS: The effectiveness of the proposed method was demonstrated in phantom experiments. Verification of its effectiveness in real organs is necessary in future study.

Audit-Based Quality Validation of the Japan Cardiovascular Surgery Database.

BACKGROUND: The Japan Cardiovascular Surgery Database (JCVSD) is a nationwide registry of patients undergoing cardiovascular surgery in Japan. To investigate and improve data quality, we have been conducting on-site institutional audits since 2004. This study aimed to investigate the accuracy of the registered data by comparing it to site visit data.Methods and Results:The subjects of this study were the 95 facilities at which a site visit was conducted. The case registration accuracy was 98.74%. Furthermore, we confirmed high data input accuracy of >90% for almost all fields. Approximately 99% of cases had been correctly entered for diabetes, aortic stenosis, and mortality. We also discovered which fields were more likely to be incorrectly captured and the causes thereof, as well as problems regarding some definitions and the input system itself. CONCLUSIONS: We were able to confirm high registration accuracy in the JCVSD. Appropriately resourced, focused site visits as part of a national audit are capable of accurate data collection on which continual nationwide quality control can be based. Continued work and development to further improve the quality of the database are mandatory to maintain a high standard of cardiovascular surgery in Japan.

Deformation of the Pelvic Arteries Caused by Pneumoperitoneum and Postural Changes in an Animal Model.

BACKGROUND/AIM: We investigated pelvic arterial deformation and shift due to intraoperative pneumoperitoneum and postural changes in an animal model. MATERIALS AND METHODS: Computed tomography images of pigs were acquired in different body positions (supine, head down at 5° and 10°, right lateral recumbent at 5° and 15°) before and after insufflation. We used a free software (3D Slicer) for image analysis. After landmark registration using 10 markers inserted into the pelvis, pelvic arterial deformation and shift of seven arterial bifurcation points were evaluated. The distance moved was the target registration error (TRE) from the points registered in the supine position. Fiducial registration error (FRE) was measured using the 10 pelvic markers. RESULTS: TRE average from postural changes ranged from 0.7 to 1.2 mm and was 1.4 mm due to pneumoperitoneum. TRE and FRE averages were 2.1 mm and 0.2 mm, respectively. CONCLUSION: The pelvis was useful for registering anatomical landmarks.

Surgical navigation system for laparoscopic lateral pelvic lymph node dissection in rectal cancer surgery using laparoscopic-vision-tracked ultrasonic imaging.

BACKGROUND: Laparoscopic lateral pelvic lymph node dissection (LPLND) in rectal cancer surgery requires considerable skill because the pelvic arteries, which need to be located to guide the dissection, are covered by other tissues and cannot be observed on laparoscopic views. Therefore, surgeons need to localize the pelvic arteries accurately before dissection, to prevent injury to these arteries. METHODS: This report proposes a surgical navigation system to facilitate artery localization in laparoscopic LPLND by combining ultrasonic imaging and laparoscopy. Specifically, free-hand laparoscopic ultrasound (LUS) is employed to capture the arteries intraoperatively in this approach, and a laparoscopic vision-based tracking system is utilized to track the LUS probe. To extract the artery contours from the two-dimensional ultrasound image sequences efficiently, an artery extraction framework based on local phase-based snakes was developed. After reconstructing the three-dimensional intraoperative artery model from ultrasound images, a high-resolution artery model segmented from preoperative computed tomography (CT) images was rigidly registered to the intraoperative artery model and overlaid onto the laparoscopic view to guide laparoscopic LPLND. RESULTS: Experiments were conducted to evaluate the performance of the vision-based tracking system, and the average reconstruction error of the proposed tracking system was found to be 2.4 mm. Then, the proposed navigation system was quantitatively evaluated on an artery phantom. The reconstruction time and average navigation error were 8 min and 2.3 mm, respectively. A navigation system was also successfully constructed to localize the pelvic arteries in laparoscopic and open surgeries of a swine. This demonstrated the feasibility of the proposed system in vivo. The construction times in the laparoscopic and open surgeries were 14 and 12 min, respectively. CONCLUSIONS: The experimental results showed that the proposed navigation system can guide laparoscopic LPLND and requires a significantly shorter setting time than the state-of-the-art navigation systems do.

Preliminary study of eye tracking to investigate the differences in gaze behaviors depending on the experience of neuroendovascular therapy.

BACKGROUND: Neuroendovascular therapy is now the choice for the management of many neurovascular pathologies, and physicians with endovascular skills are in high demand. In addition to the traditional method of practicing hand movements to learn skills, a new strategy of practicing eye movements to learn skills is also attracting attention. This preliminary study explored the differences in gaze behavior depending on experience with endovascular procedures to be facilitated in future skill training in neuroendovascular therapy. METHODS: Four physicians with experience of 3-412 neuroendovascular procedures wore eye-tracking devices during coil embolization of swine cervical arteries. Gaze metrics with direct correlations to the expertise of endovascular procedures were explored. RESULTS: Gaze metrics with a positive direct correlation to experience included the proportion of fixation durations (PFD) in the screen area and the native images. Those with a negative direct correlation included the PFD in the off-screen area and the roadmap images and the average fixation durations in the off-screen and coil areas. During the parent artery occlusion procedure with detachable coils, more experienced operators preferred to look at native images rather than roadmap images and that less experienced operators tended to look down at their hands more frequently. CONCLUSION: This preliminary study demonstrated the feasibility of eye tracking to identify the differences in gaze behavior depending on the experience of endovascular procedures and may guide future eye-tracking studies in neuroendovascular therapy.

Correction to: Thoracic and cardiovascular surgery in Japan in 2016 : Annual report by The Japanese Association for Thoracic Surgery.

In the original publication of the article, the values of the row "Norwood procedure", under "(3) Main procedure" in Table 3 were published incorrectly. The corrected part of the table is given in this Correction.

Current status of cardiovascular surgery in Japan, 2013 and 2014: A report based on the Japan Cardiovascular Surgery Database (JCVSD). 1: Mission and history of JCVSD.

The Japan Cardiovascular Surgery Database (JCVSD) was created in 2000 with the support of the Society of Thoracic Surgeons (STS). The STS database content was translated to Japanese using the same disease criteria and in 2001, data entry for adult cardiac surgeries was initiated online using the University Hospital Medical Information Network (UMIN). In 2008, data entry for congenital heart surgeries was initiated in the congenital section of JCVSD and preoperative expected mortality (JapanSCORE) in adult cardiovascular surgeries was first calculated using the risk model of JCVSD. The Japan Surgical Board system merged with JCVSD in 2011, and all cardiovascular surgical data were registered in the JCVSD from 2012 onward. The reports resulting from the data analyses of the JCVSD will encourage further improvements in the quality of cardiovascular surgeries, patient safety, and medical care in Japan.

Simultaneous evaluation of wall motion and blood perfusion of a beating heart using stereoscopic fluorescence camera system.

In this study, we aimed to develop a stereoscopic fluorescence camera system for simultaneous evaluation of wall motion and tissue perfusion using indocyanine green (ICG) fluorescence imaging. The system consists of two high-speed stereo cameras, an excitation lamp, and a computer for image processing. Evaluation experiments demonstrated that the stereoscopic fluorescence camera system successfully performed the simultaneous measurement of wall motion and tissue perfusion based on ICG fluorescence imaging. Our system can be applied to intraoperative evaluation of cardiac status, leading to an improvement in surgical outcomes.

Coronary artery bypass grafting in hemodialysis-dependent patients: analysis of Japan Adult Cardiovascular Surgery Database.

BACKGROUND: Perioperative risk during coronary artery bypass grafting (CABG) is reportedly high in patients with chronic renal disease. We aimed to determine postoperative mortality and morbidity and identify the perioperative risk factors of mortality during CABG in hemodialysis (HD)-dependent patients. METHODS AND RESULTS: From the Japan Adult Cardiovascular Surgery Database, we compared 1,300 HD-dependent chronic renal failure patients with 18,387 non-HD patients who all underwent isolated CABG between January 2005 and December 2008. The operative mortality and mortality, including major morbidity, was 4.8% vs. 1.4% and 23.1% vs. 13.7% in the HD and non-HD groups, respectively. Preoperative predictors of operative mortality included age, chronic obstructive pulmonary disease, peripheral arterial disease, congestive heart failure, arrhythmia, preoperative inotropic agent requirement, New York Heart Association class IV, urgent or emergency operation, poor left ventricular function, aortic valve regurgitation (>2), and mitral valve regurgitation (>3). Postoperative predictors of operative mortality included stroke, infection, prolonged ventilation, pneumonia, heart block, and gastrointestinal complications. CONCLUSIONS: Compared with non-HD patients, CABG in HD patients was associated with high mortality and morbidity rates. An appropriate surgical strategy and careful perioperative assessment and management for prevention of respiratory and gastrointestinal complications might contribute to improved clinical outcomes after CABG in these patients.

Endoscope system with plasma flushing and coaxial round jet nozzle for off-pump cardiac surgery.

BACKGROUND: To develop a new endoscope for performing simple surgical tasks inside the blood-filled cardiac atrium/chamber, that is, "off-pump" cardiac surgeries. METHODS: We developed the endoscope system with plasma flushing and coaxial round jet nozzle. The "plasma flushing" system was invented to observe the interior of the blood-filled heart by displacing blood cells in front of the endoscope tip. However, some areas could not be observed with simple flushing of the liquid because the flushed liquid mixed with blood. Further, a large amount of liquid had to be flushed, which posed a risk of cardiac damage caused by excess volume. Therefore, to safely capture high-resolution images of the interior of the heart, an endoscope with a coaxial round jet nozzle through which plasma is flushed has been developed. And to reduce the volume of flushed liquid, the synchronization system of heartbeat and the endoscope system with plasma flushing has been developed. RESULTS: We conducted an in vivo experiment to determine whether we could observe intracardiac tissues in swine without the use of a heart-lung machine. As a result, we successfully observed intracardiac tissues without using a heart-lung machine. By using a coaxial nozzle, we could even observe the tricuspid valve. Moreover, we were able to save up to 30% of the flushed liquid by replacing the original system with a synchronization system. And we evaluated the performance of the endoscope with the coaxial round jet nozzle by conducting fluid analysis and an in vitro experiment. CONCLUSION: We successfully observed intracardiac tissues without using a heart-lung machine. By using a coaxial nozzle, we could even observe the tricuspid valve. And by replacing an original system to a synchronization system, we were able to save up to 30% of the flushed liquid. As a follow-up study, we plan to create a surgical flexible device for valve disease that can grasp, staple, and repair cardiac valves by endoscopic visualization.